Inspection apparatus method and apparatus comprising selective frame output

ABSTRACT

A method of operating an inspection device includes collecting a plurality of successive image frames using an image sensor of the inspection device and displaying the plurality of successive image frames on a display of the inspection device. The method includes processing, via a processor of the inspection device, each image frame of the plurality of successive image frames by determining a motion parameter of each respective image frame and adding each respective image frame to a frame buffer when the respective image frame is motion free. The method includes receiving a control signal from a user interface of the inspection device requesting an image frame output. The method further includes determining, via the processor of the inspection device, a noise-reduced image frame from the frame buffer in response to the control signal and outputting the noise-reduced image frame in response to the control signal.

CROSS REFERENCE TO RELATED APPLICATIONS

Under 35 U.S.C. §120, this application is a continuation of U.S. patentapplication Ser. No. 14/292,648 filed on May 30, 2014 and a continuationof U.S. patent application Ser. No. 14/331,084 filed on Jul. 14, 2014,which is a continuation of U.S. patent application Ser. No. 11/642,569filed Dec. 20, 2006, now U.S. Pat. No. 8,810,636, which are allincorporated by reference herein in their entirety for all purposes.

FIELD OF THE INVENTION

The invention relates to inspection apparatuses generally and moreparticularly to visual inspection apparatuses.

BACKGROUND OF THE PRIOR ART

Inspection apparatuses can be used to develop streaming video imagerepresentation of areas to be inspected. In one embodiment an inspectionapparatus can be used to inspect industrial equipment articles. Atcertain times during operation of an inspection apparatus an inspectormay initiate a freeze frame control signal. When a freeze frame controlsignal is initiated, a buffered frame of image data retained in a framebuffer can continually be read out to a display. At other times duringoperation of an inspection apparatus an inspector may initiate a saveframe control signal. When a save frame control signal is initiated, abuffered frame of image data retained in a frame buffer can be writtento a memory location of a memory device for later use, e.g., a volatilememory device, a non-volatile memory device, or to a long term storagedevice.

In typical operation the first frame of image data having a captureinitiation time subsequent to the time of initiation of a control signalto output a frame of image data to a display or memory is the frame thatis subject to output. Unfortunately, the frame having the first captureinitiation time subsequent to an initiation of a control signal tooutput a frame is not always a high quality frame of image data. If theinspection apparatus is being moved at the time of initiation of acontrol signal to output a frame of image data, low quality image may besaved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram illustrating a method that can be carried outwith one of an inspection apparatus.

FIG. 2 is as electrical block diagram illustrating an exemplary set ofcircuits that can be incorporated in an inspection apparatus.

FIG. 3 is an alternative physical form view of an inspection apparatus.

FIG. 4 is an alternative physical form view of as inspection apparatus.

FIG. 5 is alternative electrical block diagram illustrating an exemplaryset of circuits that can be incorporated in an inspection apparatus.

FIG. 6 is an alternative physical form view of an inspection apparatus.

FIG. 7 is an alternative physical form view of an inspection apparatus.

FIG. 8 is a timing diagram illustrating a timing of an initiation of acontrol signal to selectively output a frame of image data plottedagainst frame capture times.

FIG. 9 is a view of an inspection apparatus having a graphical userinterface allowing selection of various frame selection algorithms thatcan be executed by an inspection apparatus.

FIG. 10 is a plot illustrating application of non-uniform digital gainin one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

A simplified flow diagram illustrating a process for selectivelyoutputting a high quality frame of image data is shown in FIG. 1. Atblock 10, a control signal to selectively output a frame of image datacan be initiated. At block 20, apparatus 100 can process image data ofone or more frames and can determine a motion parameter. In oneembodiment, apparatus 100 can determine a motion parameter for eachframe of image data processed. At block 30 apparatus 100 can selectivelyoutput a frame of image data responsively to processing at block 20. Byselectively outputting a frame of image data subsequently to a frameretention control signal being initiated, the quality of the frame ofimage data that can be output can be improved. A technical effect of theprocessing described with reference to the flow diagram of FIG. 1 is toimprove the quality of an output frame of image data. Accordingly, thereis described herein, in one embodiment, a method for operating aninspection apparatus having an elongated inspection tube and an imagesensor for generating image signals, said method comprising the stepsof: initiating a control signal to selectively output a frame of imagedata; processing image data of one or more frames to determine a motionparameter; and subsequent to initiation of a control signal toselectively output a frame, outputting a frame of image dataresponsively to said processing to determine a motion parameter. Inoutputting a frame, an inspection apparatus can output to a displayand/or an addressable memory location for later use a frame retained ina frame buffer that is continually written over during the course ofoperation of the apparatus. The frame buffer can be, e.g., an inputframe buffer, an output frame buffer, or an accumulator frame buffer.The memory location to which the buffered frame can be written to can bea memory location of a memory device, e.g., a volatile memory device, anon-volatile memory device, or a storage device.

In some embodiments, image data that is subject to processing is frameimage data captured subsequent to the time of initiation of a controlsignal to selectively output a frame of image data. In otherembodiments, image data that is subject to processing is frame imagedata captured prior to the time of initiation of a control signal toselectively output a frame of image data. In other embodiments, frameimage data subject to processing can comprise both frame image data ofsingle frames having capture initiation times of subsequent to and priorto the time of initiation of a control signal to selectively output aframe of image data. Frame image data that is subject to processing fordetermining a motion parameter can comprise a full frame of image dataor less than a full frame of image data. For example, in one embodimenta limited number of rows of image data can be processed. In anotherembodiment, a limited number of columns can be processed. In anotherembodiment, where image sensor 132 is provided to be capable of aninterlaced readout mode, a single field (e.g., an odd-field or an evenfield) can be processed. When capturing a frame of image data, apparatus100 need not simultaneously retain each image data element of a frame ofimage data. For example, when capturing a frame of image data subject toprocessing, apparatus 100 may buffer a limited amount of frame imagedata (e.g., pixel values corresponding to a few rows of pixels) at agiven instant in time.

A motion parameter can be developed for each frame subject toprocessing. A motion parameter that can be developed for each framehaving image data subject to processing can comprise a binary two-stateparameter, i.e., a frame can be designated as being “in motion” or“motion free.” A motion parameter that is developed for each frame ofimage data subject to processing at block 20 can, in addition or in thealternative, comprise a qualitative measurement of motion. For example,a motion score can be ascribed to each frame of image data correspondingto the degree of motion determined to be present in the frame of imagedata.

A block diagram of an exemplary apparatus capable of supporting theabove described processing is shown and described in connection withFIG. 2. Inspection apparatus 100 can include an elongated inspectiontube 112 and a head assembly 114 disposed at a distal end of theelongated inspection tube. Head assembly 114 can include solid stateimage sensor 132 and imaging lens 140. Imaging lens 140 can focus animage onto an active surface of solid state image sensor 132. Imaginglens 140 can comprise, e.g., a lens singlet or a lens having multiplecomponents, e.g., a lens doublet, a lens triplet. Solid state imagesensor 132 can be, e.g., a CCD or CMOS image sensor. Solid state imagesensor 132 can include a plurality of pixels formed in a plurality ofrows and columns. Solid state image sensor 132 can provided on anintegrated circuit. Image sensor 132 can generate image signals in theform of analog voltages representative of light incident on each pixelof the image sensor. Referring to further aspects of head assembly 114,image sensor 132 can be controlled so that image signals are clocked outfrom image sensor 132. Analog voltages representative of light incidenton the various pixels of image sensor 132 can be propagated throughsignal conditioning circuit 136 along a cable, e.g., a coaxial cabledisposed within elongated inspection tube 112. Head assembly 114 caninclude signal conditioning circuit 136 when conditions analog imagesignals for input to cable 138 and receives timing and control signalsfor control of image sensor 132. In one embodiment, image sensor 132 andsignal conditioning circuit 136 can be mounted on a common circuit board137. In the embodiment of FIG. 2 an imaging axis 250 of apparatus 100extends through head assembly 114.

In the embodiment of FIG. 2, head assembly 114 of apparatus 100 at adistal end of inspection tube 112 comprises image sensor 132. Imagesensor 132 of inspection apparatus 100 can, in one alternativeembodiment be located at a position spaced apart from head assembly 114,and disposed at a position rearward of a proximal end of inspection tube112. For example, image sensor 132 can be disposed in base assembly 105interfaced to elongated inspection tube 112 as shown in FIG. 2. Animaging system fiber optic bundle (not shown) can be disposed inelongated inspection tube 112, and can terminate in head assembly 114.The apparatus can be configured so that such a fiber optic bundle relaysimage forming light rays from head assembly 114 to the spaced apartimage sensor spaced apart from head assembly 114.

Various circuits disposed at a position spaced apart from head assembly114 can receive and process image signals generated by image sensor 132.In one embodiment, various circuits receiving and processing imagesignals generated by image sensor 132 can be disposed in base assembly105 interfaced to elongated inspection tube 112 as shown in FIG. 2. Inthe exemplary embodiment of FIG. 2, analog front end circuit 150 caninclude an analog gain circuit, an analog-to-digital converter, and acorrelated double sampler and can receive analog image signals, digitizesuch signals, and transmit digitized image signals to digital signalprocessor 152 (DSP). DSP 152, in the embodiment shown, can be configuredto perform such processing tasks as color matrix processing, gammaprocessing, and can process digital image signals into a standardizedvideo format, wherein video signals are expressed in a standardized dateformat. By way of example, video signals output by DSP 152 can be in aBT656 video format and data carried in the video signal can have a422YCRCB data format. DSP 152 can be in communication with a randomaccess memory 160 through system bus 158. Referring to further aspectsof an electrical circuit for inspection apparatus 100, apparatus 100 caninclude timing generator circuit 156 which can send timing and controlsignals to signal conditioning circuit 136 for input to image sensor 132as well as to analog front end circuit 150 and DSP 152. As indicated bycommunication line labeled “to 136,” timing generator circuit 136 cansend control signals such as exposure timing signals frame rate timingsignals to signal conditioning circuit 136 for input to image sensor132. In one embodiment, analog circuit front end 150, DSP 152, andtiming generator circuit 156 can be provided on separate integratedcircuits (ICs). In one embodiment, analog front end circuit 150, DSP152, and tinting generator circuit 156 are provided as part ofcommercially available chips, e.g., an SS2 DSP chipset of the typeavailable from SONY. While an analog to digital converter for convertinganalog image signals into digital form is described as beingincorporated into front end circuit 150, such an analog to digitalconverter can be incorporated into an image sensor integrated circuitwhich commonly carries pixels of an image sensor and an analog todigital converter for digitizing analog image signals.

Referring to further aspects of apparatus 100, apparatus 100 can includeDSP 180. DSP 180 can receive the formatted video output from DSP 152 forfurther processing. DSP 180 can be configured to perform a variety ofprocessing tasks such as frame averaging, scaling, zoom, overlaying,merging, image capture, flipping, image enhancement, and distortioncorrection. DSP 180 can also be configured to perform motion detectionas will be described more fully herein. In one embodiment, DSP 180 canbe provided by a TMS32ODM642 Video/Imaging Fixed-Point Digital SignalProcessor of the type available from TEXAN INSTRUMENTS. DSP 180 can bein communication with a volatile memory 161, e.g., RAM, a non-volatilememory 162, and storage memory device 164. Non-volatile memory 162 canbe provided, e.g., by a flash memory device, an EEPROM memory device, oran EPROM memory device. Software for operating apparatus 100 can beretained in non-volatile memory 162 when apparatus 100 is not operatingand such software can be loaded into RAM 161 when apparatus 100 isdriven into an operating state. Apparatus 100 can include other types ofstorage memory. For example, a USB “thumb drive” can be plugged intoserial I/O interface 172. A CompactFlash memory card can be plugged intoparallel I/O interface 173. A memory of apparatus 100 can be regarded asincluding memory 161, 162, and 164, other storage memory, as well asinternal buffer memories of DSP 152 and 180. Storage memory device 164can be, e g., a hard drive or removable disk. RAM 161, non volatilememory 162, and storage device 164 can be in communication with DSP 180via system bus 159. While DSP 152 and DSP 180 are shown as beingprovided on separate integrated circuits, the circuits of DSP 152 andDSP 180 could be provided on a single integrated circuit. Also, thefunctionalities provided by DSP 152 and DSP 180 could be provided by oneor more general purpose microprocessor IC.

Apparatus 100 can be configured so that image signals are read out ofimage sensor 132 row by row until a frame of image signal includingimage signals corresponding to multiple pixels of image sensor 132 havebeen read out. Analog image signals read out from image sensor 132 am beconverted into digital form by front end circuit 150. Front end circuit150, in turn, can feed digitized frame image signals into DSP 152. DSP152 can format the image signals into a specific format before feedingthe digitized image signals for further processing to DSP 180. Digitizedframe image signals can be referred to as frame image data.

Referring to further circuit components of the block diagram of FIG. 2,apparatus 100 can further include display 210, keyboard 214, andjoystick 218. Keyboard 214 enables a user to initiate various controlsignals for the control of apparatus 100. Display 210 enables display oflive video streaming images and other images to an inspector. Forexample, apparatus 100 can be controlled to switch from a live streamingvideo mode in which a live streaming video is being displayed on display210 to a mode in which a still image is displayed on display 210.Apparatus 100 can be configured so that apparatus 100 can generatecontrol signals to selectively output a frame responsively to an actionby an inspector. Apparatus 100 can be configured so that an inspectorcan initiate a control signal to selectively output a frame of imagedata by actuating a designated button of keyboard 214. Control signalsfor selective output of a frame of image data can include, e.g., afreeze frame control signal, and a save frame control signal. Apparatus100 can be configured so that when a freeze frame control signal isinitiated, apparatus 100 can repeatedly (continuously) output a framefrom a frame buffer to display 210. The frame buffer can be continuouslyoverwritten during the course of operation of the apparatus. The framebuffer can be a buffer of RAM 161, and can be, e.g., an input framebuffer, an output frame buffer, or an accumulator frame buffer.Apparatus 100 can be configured so that when a “save frame” controlsignal is initiated, apparatus 100 can output a frame from a framebuffer to an addressable memory location for future access., e.g., amemory location of RAM 161, non-volatile memory 162 and/or storagedevice 164. A frame of image data saved responsively to initiation of asave frame control signal can be formatted into a standardized or knownproprietary file format. During performance of an inspection procedure,an inspector may initiate a save frame control signal several times tosave numerous frames relating to a work subject (e.g., an equipmentarticle) being subject to an inspection. A sensor interface of apparatus100 can include keyboard 214, joystick 218, and display 210.

In a further aspect, DSP 180 can be coupled to a serial I/O interface172, e.g., an ETHERNET or USB interface and a parallel data interface,e.g., a CompactFlash interface or PCMCIA interface. DSP 180 can also becoupled to a wireless data communication interface 174, e.g., an IEEE802.11 interface. Apparatus 100 can be configured to send frames ofimage data saved in a memory thereof to an external computer and canfurther be configured to be responsive to requests for frames of imagedata saved in a memory device of apparatus 100. Apparatus 100 canincorporate a TCP/IP communication protocol suite and can beincorporated in a wide area network including a plurality of local andremote computers, each of the computers also incorporating a TCP/IPcommunication protocol suite. With incorporation of TCP/IP protocolsuite, apparatus 100 incorporates several transport layer protocolsincluding TCP and UDP and several different layer protocols includingHTTP and FTP.

Referring to further aspects of apparatus 100, apparatus 100 can includejoystick 218 for controlling a positioning of head assembly 114. In oneembodiment, articulation cables 222 can be incorporated in elongatedinspection tube 112 to enable movement of head assembly 114 into adesired position so that a field of view of apparatus 100 can bechanged. Joystick 218 can be in communication with DSP 180. Apparatus100 can be configured so that control signals for controlling movement(articulation) of head assembly 114 are initiated by manipulatingjoystick 218. Apparatus 100 can be configured so that when joystick 218is moved, DSP 180 receives a control signal from joystick 218 and sendscorresponding motor control signals to articulation motor 220 to producea desired movement of head assembly 114. Apparatus 100 can also beconfigured so that joystick 218 operates as a pointer controller forcontrolling a pointer displayed on display 210.

In another aspect, inspection apparatus 100 can include a light source230, (e.g., an arc lamp or a bank of one or more LEDs), which, likecircuits 150, 152, 156, and 180 can be disposed at a position spacedapart from head assembly 114. Apparatus 100 can also include anillumination fiber optic handle 232 receiving light emitted from lightsource 230. Fiber optic bundle 232 can be disposed in elongatedinspection tube 112 so that fiber optic bundle 232 can relay lightemitted from light source 230 through inspection tube 112 and to headassembly 114. A distal end of fiber optic bundle 232 can be interfacedto diffuser 234 for diffusing illumination light. Fiber optic bundle 232and diffuser 234 can be arranged to project light over an areaapproximately corresponding to a field of view of image sensor 132. In afurther aspect, light source 230 can be powered by a regulator 248coupled to a power supply circuit 250. Power supply circuit 250 can bearranged to power circuit board 252 receiving various integratedcircuits of apparatus 100 as well as buses 158, 159. Power supplycircuit 250 can be interfaced to various alternative power sources,e.g., serial I/O power source 254, AC/DC transformer source 256 andrechargeable battery 258.

During operation to output a live streaming video image on display 210,incoming frames may be input into an input frame buffer of RAM 161,subject to processing by DSP 180 and output to an output frame buffer ofRAM 161. Apparatus 100 can be configured so that when a freeze framecontrol signal is initiated, a frame of an output frame buffer iscontinually output to display 210. Apparatus 100 can also be configuredso that when a save frame control signal is initiated, a frame of aninput frame buffer is output to an addressable memory location of amemory device, e.g., RAM 161, non-volatile memory 162, or long termstorage device 164.

Exemplary physical form views of the apparatus 100 shown in anelectrical block view of FIG. 2 are shown in FIGS. 3 and 4. In the viewof FIG. 3, apparatus 100 includes elongated inspection tube 112, andhandset 101 incorporating housing 102, display 210, keyboard 214, andjoystick 218. Circuits 150, 152, 156, 158, 160, 162, 164, 172 and 180can be incorporated in housing 102. In the embodiment of FIG. 4,apparatus 100 includes a base unit 103 having a housing 104incorporating a subset of the circuits shown in FIG. 2. For example,housing 104 can incorporate circuits 162, 164, 180, and 172. Handset 101of FIGS. 3 and 4 can be a hand held handset sized and shaped to be heldin a human hand. Skilled artisans will recognize that modifications tothe circuit of FIG. 2 may be required if the circuits therein aredescribed between a plurality of housings. For example,serial-deserializer circuits and twisted pair couplings as are explainedin U.S. Provisional Patent Application No. 60/786,829 filed Mar. 27,2006, incorporated herein by reference can be employed to transmitrequired video and control signals over distances of several feet at ahigh data rate. Additional circuits might be employed for communicatinguser initiated control signals generated at handset 101 to base unit103. Additional circuits might also be employed for communicating imagesignals from base unit 103 to handset 101.

In one embodiment, apparatus 100 can have a base assembly 105,incorporating the components designated within dashed-in border 105 ofFIG. 2. The components of base assembly 105 can be spread out into oneor more housings. In the embodiment of FIG. 3, a single housing baseassembly is provided. In the embodiment of FIG. 4, base assembly 105comprises handset 101 and base unit 103. In another embodiment (notshown), base assembly 105 can include handset 101 and base unit. 103.However, rather than being interfaced to handset 101, elongatedinspection tube 112 can be interfaced to base unit 103.

While methods described herein can be carried out utilizing aninspection apparatus having an elongated inspection tube, methodsdescribed herein can be carried out utilizing an inspection apparatusother than inspection apparatuses having an elongated inspection tube.In FIG. 5 there is shown an inspection apparatus 100 devoid of anelongated inspection tube. In the embodiment of FIG. 5, apparatus 100 isprovisioned similarly to the embodiment of FIG. 2 except that imaginglens 140 as well as image sensor 132, signal conditioning circuit 136,and circuit board 137 are incorporated in base assembly 105. In theembodiment of FIG. 5, inspection apparatus 100 can include zoom lensmotor 224 for varying a focal distance of inspection apparatus 100.

Base assembly 105, in the embodiment of FIG. 5, can take on a variety offorms. In the embodiment of FIG. 6 showing an inspection apparatus inthe form of a hand held digital camera, base assembly 105 is provided bya hand held housing 102. The embodiment of FIG. 6 is similar to theembodiment of FIG. 3 except that whereas in the embodiment of FIG. 3imaging lens 140 as well as image sensor 132, signal conditioningcircuit 136 and circuit board 137 are incorporated in head assembly 114,imaging lens 140 as well as image sensor 132, signal conditioningcircuit 136 and circuit board 137 in the embodiment of FIG. 6 areincorporated in base assembly 105 which in the embodiment of FIG. 6 isprovided by a hand held housing 102. In the embodiment of FIG. 6,imaging axis 150 of apparatus 100 extends through hand held housing 102.In the embodiment of FIG. 7, inspection apparatus 100 is provided in theform of a pan-tilt-zoom (PTZ) camera. A PTZ camera as shown in FIG. 7can be adapted to be mounted on a flat surface such as a ceiling, wall,table, or such as may be provided by a mounting platform of a robot. APTZ camera as shown in FIG. 7 can be used in a variety of inspectionapplications such as robot inspections and surveillance monitoring. Inthe embodiment of FIG. 7, circuit components can be incorporated asshown in FIG. 5 such that imaging lens 140 as well as image sensor 132,signal conditioning circuit 136, and circuit board 137 are incorporatedin base assembly 105 provided as shown in FIG. 7 by PTZ camera housing106. In the embodiment of FIG. 7, imaging axis 250 can extend through acamera housing 106 as shown in the embodiment of FIG. 7. Referring stillto the embodiment of FIG. 7 which can incorporate the circuitdistribution of FIG. 5, inspection apparatus 100 can incorporate motorassembly 222 for controlling a pan and tilt of the inspection apparatuswhen provided by an inspection apparatus in the form of a PTZ camera.Keyboard 214, display 210, and joystick 218 (pointer controller) can beprovided on board PTZ camera housing 106 as shown in FIG. 7, or else maybe distributed into an inspection apparatus housing spaced apart fromPTZ camera housing 106. As indicated by dashed-in laptop PC housing 107of FIG. 7, circuits of FIG. 5 can be distributed into housingsextraneous from housing 106. A PC incorporated in housing 107 caninclude various circuits such as DSP 180 and other circuits and can beconfigured to perform various image processing methods as describedherein. A PC incorporated in housing 107 can be connected to the PTZcameral incorporated in housing via IP network 109. Inspection apparatus100 can also be provided by a camera of a machine vision system for usein an assembly process or other industrial process.

An inspection apparatus as described in connection with FIG. 2 can beconfigured to perform a method as described in connection with the flowdiagram of FIG. 1. Accordingly, there is described herein, in oneembodiment an inspection apparatus comprising: an elongated inspectiontube; a two dimensional image sensor comprising a plurality of pixels,the two dimensional image sensor generating image signals correspondingto light incident on said pixels, wherein said inspection apparatus isconfigured to be capable of: (i) responding to a user-initiated controlsignal to selectively output a frame of image date; (ii) processingimage data of one or more frames to determine a motion parameter; and(iii) outputting a frame of image data responsively to said processing.

Referring again to the flow diagram of FIG. 1, further details andvariations of a method that can be performed with use of apparatus 100are described. As described to with reference to the block diagram ofFIG. 2, a user can initiate a control signal to selectively output aframe of image data by actuating a button of keyboard 214. Apparatus 100can be configured to responsively generate selective frame outputcontrol signal responsively to an action of an inspector; e.g.,actuating a button of keyboard 214.

At block 20, apparatus 100 can determine a motion parameter for each ofat least one frame of image data. Apparatus 100, in one embodiment canbe configured to execute block 20 subsequent to a time at which acontrol signal to selectively output a frame of image data at block 10is initiated. Apparatus 100, in another embodiment can be configured sothat apparatus 100 is executing processing block 20 at the time at whicha control signal to selectively output a frame of image data isinitiated. Apparatus 100 can be configured to execute processing block20 in a number of different ways. For example, apparatus 100, indetermining a motion parameter for each of one or more frames, candetermine a binary (motion or motion free) motion parameter for a frameor can determine a qualitative motion parameter indicative of a degreeof motion (e.g., can develop a motion scale from 0 to 9 where 0 is nomotion and a 9 is maximum motion). Apparatus 100 can output a frame ofimage data responsively to initiation of a control signal to selectivelyoutput a frame of image data in a number of alternative possible ways.In one example, apparatus 100 can determine a motion parameter for a setof one to N frames. The set of one to N frames can be a set ofsuccessively captured frames of image data. Since the set of one to Nframes can be captured within a certain time window, the set of one to Nframes can be regarded as a “window.” In FIG. 8, there are showntimelines 270, 272 illustrating a time of initiation of a control signalto selectively output a frame of image data plotted against framecapture times. Timeline 270 shows a time of initiation of a framecontrol signal to selectively output a frame (e.g., a freeze frame or asame frame control signal). In the specific embodiment of FIG. 8, thetime of initiation of a control signal is referenced by referencenumeral 271. Timeline 272 shows times at which single frames (i.e.,frames having image data corresponding to a specific frame readoutperiod) of image data are captured. Apparatus 100 can be configured tocontinuously capture frames of image data as indicated by timeline 272,wherein the leading edges are the times where a first pixel value for aframe is buttered in DSP 180 and the falling edges are the times where alast pixel value of a frame is buffered by DSP 180. Referring to FIG. 8,apparatus 100 can determine a motion parameter for one or more framesafter a time of initiation of a control signal to selectively output aframe, (i.e., frames within window 281). However, as indicated,apparatus 100 can be processing frames of image data to ascribe a motionparameter score to each frame of image data processed prior to time thata frame control signal to selectively output a frame is initiated. Itshould be noted that in buffering a frame of image data, a frame bufferof apparatus 100 such as a buffer of DSP 180 need not buffer each pixelvalue making up a frame of image data simultaneously. A frame buffer maybe used to buffer only a subset of pixel values (e.g., a few rows)making up a frame of image data at a given time.

In one embodiment, each frame of image data that is captured byapparatus 100 prior to a time of initiation of a control signal toselectively output a frame can be subject to processing to determine amotion parameter. In one embodiment a set of frames subject to motionparameter processing comprises only frames (i.e., frames of window 282)captured prior to the time of 219799-4 initiation of a control signal toselectively output a frame. In another embodiment, a set of framessubject to motion parameter processing comprises both frames capturedprior to the time of initiation of a control signal to selectivelyoutput a frame and frames captured subsequent to the time of initiationof a frame control signal to selectively output a frame, i.e., frames ofwindow 283 in one example. In another embodiment, only frames of imagedata captured after initiation of a control signal to selectively outputa frame are subject to processing.

Rules for determining which frame of N frames subject to processing isto be selectively output can be varied. Where apparatus 100 develops abinary motion parameter, apparatus 100 can selectively output the firstframe subject to processing having a “motion free” designation. Whereapparatus 100 develops a motion parameter score, apparatus 100 can,after processing a set of N frames, selectively output the frame havingthe lowest motion score. When processing image data of a set of N framesto selectively output a frame having a lowest motion score, apparatus100 can buffer each incoming frame to a designated frame buffer locationunless the incoming frame has a higher motion score that the currentlybuffered frame, in which case the incoming frame can be discarded. Insuch manner it is not necessary to buffer image data of N framessimultaneously when processing image data of N frames for purposes ofdetermining the frame having the lowest motion score.

A number of possible configurations for processing image data of one ormore frames and to selectively output frame responsively to theprocessing are summarized in Table A. As will be described furtherherein, apparatus 100 can be configured so that the exemplaryconfigurations are user-selective. Apparatus 100 can be configured sothat apparatus 100 activates a different algorithm for selectivelyoutputting a frame of image data depending on which configuration isactive.

TABLE A Config- Motion Parameter uration Window Developed Decision 1Process one to N Binary “motion” Selectively output frames capturedafter or “motion free” first frame initiation of a designated asselective frame motion free output control signal 2 Process N framesQualitative motion Selectively output captured prior to parameter framewith lowest initiation of a developed motion score selective frame(Score between output control signal 0-9) 3 Process N frames Binary“motion” Selectively output wherein some frames or “motion free” firstframe are captured before designated as initiation of a motion free orNth selective frame frame if no frame output control signal isdesignated as and some frames are being motion free captured afterinitiation of a selective frame output control signal 4 Process Nframes. Binary “motion” Selectively output The N frames may be or“motion free” first frame if F captured before, successive framesduring, or after are designated as initiation of a being motion freeselective frame output control signal 5 Process N frames. Qualitativemotion Selectively output The N frames may be parameter frame withlowest captured before, developed motion score during, or after (Scorebetween initiation of a 0-9) selective frame output control signal

Methods for determining motion parameters are now described. Where imagesensor 132 is of the type having an interlaced frame readout modewherein an odd field of a frame is read out and then an even field,motion can be detected for by subtracting the even field from the oddfield. The difference result can be scaled to yield a motion parameterscore, e.g., between 0 and 9 wherein 0 is a score for no motion and 9 isa score for extensive motion. When head assembly 114 is not in motion, amotion parameter score can be expected to be about 0, though diagonallines and/or horizontal edges may cause non-zero difference results.Even so, such analysis of frames including such diagonal lines and/orhorizontal edges generally yields lower difference results formotion-free frames than for frames with motion. For converting a scoreto a binary motion parameter, i.e., “in motion” or “motion free”clarification, the score can be subject to thresholding (i.e, all scoresbelow 2 are deemed to be motion free).

In another method for detecting motion, apparatus 100 can examine firstand second successively captured frames. In examining first and secondsuccessively captured frames, apparatus 100 can locate one or morecommon edges in first and second frames, and can subtract pixelpositions forming the common edge of the second frame from the firstframe to derive a motion parameter scalable to scale, e.g., from 0 to 9.When head assembly 114 is not in motion, a motion parameter degree ofmotion score can be expected to be about 0. For converting a score to abinary motion parameter, i.e., “in motion” or “motion free”classification, the score can be subject to thresholding (i.e., allscores below 2 are deemed to be motion free).

In yet another method for detecting motion, apparatus 100 can examineimage data of several frames in the form of first and secondsuccessively determined super frames. Each super frame can be determinedby processing a set of M successively captured single frames. Theprocessing can include, e.g., averaging or summing M successivelycaptured frames, In one example, with a set of 10 successively capturedframes, 0 to 9, a first super frame can be derived by averaging frames 0through 4 and the second super frame can be derived by averaging frames5 through 9. For conservation of processing requirements, accumulatorsmay be employed for averaging. Super frames can be determined on amoving window basis. For example, during a first frame period, a firstaccumulator can retain the average or sum of frames N N. . . (N+4), anda second accumulator can retain the average or sum of frames (N+5) . . .(N+9). In a next frame period, the first accumulator can retain theaverage or sum of frames (N+1) . . . (N+5) and the second accumulatorcan retain the average or sum of frames (N+6) . . . (N+10). In examiningfirst and second successively captured super frames, apparatus 100 canlocate a common edge in first and second super frames, and subtractpixel positions forming the common edge of the second super frame fromthe first super frame to derive a motion parameter scalable to scale,e.g., from 0 to 9. When head assembly 114 is not in motion, a motionparameter score can be expected to be about 0. For converting a score toa binary motion parameter, i.e., “in motion” or “motion free”classification, the score can be subject to thresholding (i.e., allscores below 2 are deemed to be motion free). In another embodiment,apparatus 100 can be configured to subtract a super frame from apreceding super frame for purposes of developing a motion parameter. Theinventor found that processing of super frames for motion detection isparticularly advantageous under lower brightness (and, therefore, underhigher expected noise) conditions. In one embodiment, apparatus 100 canbe configured to process incoming image data for detection of brightnessand can further be configured to automatically switch from a mode ofoperation in which single frames are processed for motion detection to amode in which super frames are processed for motion detection when it isdetermined that brightness has fallen below a threshold brightnesslevel.

In another embodiment, apparatus 100 when outputting a frame at block 30responsively to a processing at block 20 can output a noise reducedframe. A noise reduced frame can be provided by processing a pluralityof captured single frames as in a super frame. The plurality of framesthat can be processed for providing a noise reduced frame can besuccessive frames or non-successive frames. For example, apparatus 100,as described in connection with FIG. 9 can be configured to process aset of 128 frames (F0 . . . F127) in determining a noise reduced frame.In processing the frames, apparatus 100 can determine if the frames arein motion, and can discard frames determined to be in motion. Indiscarding a frame, apparatus 100 can avoid inputting a frame determinedto be in motion into an accumulator retaining a noise reduced frame.Apparatus 100 can also be configured to locate an edge in each frame andoffset frames of the set of frames so that located edges are aligned. Inoffsetting a frame, apparatus 100 can offset a frame prior toaccumulating the frame in an accumulator. Processing of a plurality offrames to determine a super frame can include averaging several framesor by otherwise utilizing image data from the plurality of frames toprovide a noise reduced frame. A noise reduced frame provided byaveraging a plurality of successively captured frames can be regarded asa frame averaged noise reduced frame. A noise reduced frame can also beregarded as a “filtered” frame. Where apparatus 100 is configured tooutput a noise reduced frame responsively to an initiation of a controlsignal to selectively output a frame, apparatus 100 can be configured todetermine a binary “in motion” or “motion free” motion parameterclassification for each frame of image data that is subject toprocessing. Apparatus 100 can further be configured so that when a firstframe is determined to be motion free, an accumulator begins to maintainan accumulated average frame for a next set of F frames. Apparatus 100can also be configured so that when F frames have been accumulated intothe average frame accumulator, and with each frame being motion free,and with a selective frame output control signal being initiated withina predetermined time window of F motion free frames being accumulated,the accumulated average frame determined by averaging the F frames issaved into a memory of apparatus 100.

Accordingly, there is described herein, in one embodiment, a method foroperating an inspection apparatus of the type having an elongatedinspection tube and an image sensor generating image signals, saidmethod comprising the steps of: configuring said inspection apparatus toprocess a plurality of frames to provide a noise reduced frame of imagedata; generating a control signal to selectively output a frameresponsively to an action by a user to initiate said control signal toselectively output a frame, processing image data to determine a motionparameter; and, subsequent to generation of said control signal toselectively output a frame, outputting a noise reduced frameresponsively to said processing to determine a motion parameter. Thenoise reduced frame can be output to a display and/or a memory device,e.g., device 161, 162 and/or 164.

An embodiment wherein a noise reduced frame (which can be regarded as a“filtered” frame) can be output responsively to an initiation of acontrol signal to selectively output a frame is summarized herein inTable A (See configuration 4 of Table A). In one embodiment, thecandidate configurations summarized in Table A are user selectable.

As shown in FIG. 9, apparatus 100 can be configured to displaydesignators (e.g., text or icons) corresponding to each configurationsummarized in Table A. Apparatus 100 can also be configured so that aninspector can highlight a different designator 602, 604, 606, 608, 610by moving joystick 218 and further so that an inspector can select agiven configuration by actuating a button of keyboard 214 whendesignator corresponding to a designated configuration is highlighted.The user interface of FIG. 9 having displayed designators for each ofseveral configurations can be regarded as a graphical user interface(GUI). Accordingly, there is described herein, in one embodiment, aninspection apparatus comprising: an elongated inspection tube; a twodimensional image sensor comprising a plurality of pixels, the twodimensional image sensor generating image signals corresponding to lightincident on said pixels, a user interface enabling a user to activatefirst or second configurations, the apparatus being adapted so that whensaid first configuration is active, said apparatus selects a frame foroutputting according to a first algorithm, said apparatus further beingadapted so that when said second configuration is active said apparatusselects a frame for outputting according to a second algorithm, whereinsaid second algorithm is different than said first algorithm; whereinsaid inspection apparatus is further configured to allow a user toinitiate a control signal to selectively output a frame of image data;and wherein said inspection apparatus subsequent to an initiation of acontrol signal to selectively output a frame of image data selects aframe of image data for outputting in a manner that varies depending onwhether said first configuration or said second configuration has beenselected.

In another aspect, inspection apparatus 100 can be configured to applydigital gain non-uniformly over a frame of image data. In oneembodiment, apparatus 100 can be configured to determine positiondependent digital gain parameters for pixel values of a frame of imagedata and to apply the determined position dependent digital gainparameters in determining pixel values of a frame of image data foroutputting a display and/or a memory device. The frame of image data forwhich non-uniform digital gain parameters (non-uniform and offsetparameters) can be determined can be a frame corresponding to a field ofview of apparatus 100.

Inspection apparatuses are often used to capture frames of image datarepresenting shiny surfaces. When a frame of image data representing ashiny surface is captured, illumination tends to reflect off the shinysurface causing what is often termed an over-bloomed bright spot in aframe of image data. In that bright spots will affect an overallbrightness level used to determine applied digital gain and/or exposureparameters according to an imaging parameter determining algorithm, thepresence of over-bloomed bright spots can lead to applied exposureperiod parameters and/or analog gain being too low, resulting in a frameof image data that is too dark in all but the area of an over-bloomedbright spot.

For addressing the problem of over-bloomed bright spots, inspectionapparatus 100 can be configured to apply digital gain non-uniformly overa frame of image data in order to selectively brighten a frame of imagedata in areas other than a bright spot without substantial or withoutany brightening of a frame of image data in an area about a bright spot.Inspection apparatus 100 can also be configured to apply offsetsnon-uniformly over a frame of image data in order to reduce a washouteffect of a frame of image data.

An exemplary method for outputting a frame of image data utilizing a setof position dependent non-linear digital gain values is as follows:

-   -   1. Add up luminance (e.g., gray scale) values for pixel        positions within a region surrounding each pixel position (e.g.,        a 16×16 pixel position area) to obtain a regional brightness        value for each pixel position.    -   2. Provide a lookup table mapping regional sum values to digital        gain values (parameter). The lookup table, in one embodiment,        can map larger digital gain values to smaller regional        brightness values and zero or near-zero digital gain values to        larger regional brightness values.    -   3. Determine a position dependent digital gain value utilizing        the lookup table for each pixel position.    -   4. For each pixel position multiply the original pixel value by        the determined digital gain value.

The result of applying non-uniform digital gain values determinedresponsively to the determination of regional brightness values isdescribed in greater detail with reference to the plot of FIG. 10showing pixel values through an arbitrary line of pixel positions withina bright region of pixel positions.

Referring to plot 702, line 704 indicates pixel brightness values for arow of pixels which is relatively dark at the left side and graduallybrightens across the line. At the right side of the row, the brightnessvalues clipped at the maximum possible value (e.g., 255 in an 8 bitpixel value frame), as shown by line 710. Dotted line 706 indicates awould-be pixel values if gain were applied uniformly, and bold line 708indicates pixel values where non-uniform digital gain as describedherein is applied. Referring to plot 702 it is seen with reference tothe original image data 704 that several pixel values may be clipped atthe peak 710 pixel value (indicating a possible over-bloomed brightspot). However, referring to image data 706 after application of auniform digital gain parameter, several additional pixels can be clippedat the peak pixel value, resulting in loss of contrast informationuseful to an inspector. Referring to image data 708 after application ofnon-uniform digital gain, digital gain may be applied to increase pixelvalues in the non-clipped portions of the row; however, substantially noadditional pixel values are clipped at the maximum pixel value.Accordingly, by application of the non-uniform gain parametersdetermined responsively to a determination of regional brightnessvalues, clipping of additional pixel values is substantially avoided.According to the method described herein, wherein non-uniform digitalgain parameters are determined responsively to a determination ofregional brightness values, pixel positions of relatively darker pixelvalued regions of a frame (darker regions) can have applied theretodigital gain parameters which would result in clipping of pixel valuesof pixel positions of relatively brighter pixel value regions (brighterregions) of a frame. Also, pixel positions of a relatively bright pixelvalue region of a frame can have applied thereto digital gain parameterssmaller in value than the digital gain parameters applied to therelatively darker pixel position region. Application of the relativelysmaller digital gain parameters determined responsively to adetermination of a regional brightness value within a region will resultin clipping of a fewer number of pixel values than would have beenclipped by application of uniform gain parameter sufficient to renderdarker pixel values visibly brighter. A region can be regarded herein asa set of positionally adjacent pixel positions, e.g., a block of 16×16positionally adjacent pixel positions.

In addition, another lookup table can be provided to provide mappingbetween regional sum values and a set of offset values (parameters).Such mapping can map larger offset values to smaller regional brightnessvalues and little or no offset values to larger regional brightnessvalues to reduce a “washout effect” when only digital gain is used. Forexample, when gain is applied to a frame, fine detail transitions mightbe amplified, but nevertheless, may not be rendered highly visible ifthe image data forming a transition has high white values (e.g., thehuman eye has difficulty in perceiving differences in differentiated buthigh gray scale values). For example, a human eye may have difficulty inperceiving an edge formed by an edge comprising 220 and 250 white levelpixel values (the “washout effect”). The washout effect can be addressedby applying an offset, e.g., subtracting 100 from the area of thetransition so that it is represented by pixel values having white levelsof 120 and 150. For improving a quality of an image, offset can beapplied non-uniformly by mapping pixel positions to offset parameters asindicated. For example, so that a white spot retains its appearance as awhite spot in a frame of image data having offsets applied, it would notbe desirable to have offsets applied to a white spot. By application ofnon-uniform offset pixel values of pixel positions that are inrelatively dark regions prior to application of gain can be reduced byan offset so that transitions represented therein can be rendered morevisible to an observer (e.g., an inspector). Pixel values of pixelpositions of relatively bright regions prior to application of gain canhave relatively little offset applied so that they are represented inaccordance with their original brightness levels.

In one embodiment, the frame of image data to which non-uniform digitalgain and/or offset parameters are applied can be a buffered frame ofimage data of an output frame buffer for output to display in astreaming video display.

In another embodiment, the frame of image data to which non-uniformdigital gain and/or offset parameters are applied can be a frame ofimage data output to a memory device from a frame buffer (e.g., an inputframe buffer) in response to an initiation of save frame control signal.

In another embodiment, the frame of image data to which non-uniformdigital gain and/or offset parameters can be applied can be a frame ofimage data output to a display from a frame buffer (e.g., an outputframe buffer) responsively to a processing of image data fordetermination of a motion parameter.

In another embodiment, the frame of image data to which non-uniformdigital gain and/or offset parameters can be applied can be a noisereduced frame provided by processing of several frames and retained inart accumulator buffer as described herein. A noise reduced frameprovided by processing of several single frames can be output to adisplay responsively to a processing of image data for determination ofa motion parameter. Such a noise reduced frame to which non-uniformdigital gain and/or offset parameters can be applied can also be framethat is output to a memory device in response to an initiation of a saveframe control signal. By applying non-uniform digital gain and/or offsetparameters to a frame provided by processing of several single frames,noise is reduced prior to the application of the digital gain. Thus, theapplied digital gain tends to make image details more visible withoutcreating a high noise level as may occur when digital gain is applied toa single potentially noisy frame. Additionally, where non-uniformdigital gain and/or offset parameters are applied to a frame provided byprocessing several single frames, the accumulation of frames effectivelyincreases the dynamic range available (such as from an 8-bit singleframe to a 16-bit accumulator) allowing the application of digital gainwithout reducing the number of achievable output levels as describedpreviously with uniform digital gain. It may further be desirable to usedifferent digital gain and offset tables based on the number ofaccumulated frames such that lower gains are applied when few frames areaccumulated and higher gains are applied when more frames areaccumulated. This approach minimizes the amplification of image noisewhen few frames are accumulated while allowing significant enhancementwith little noise once many frames have been accumulated. It alsoprovides a gradual transition in the image appearance which is generallypreferred over abrupt changes as would be seen if no enhancement wereapplied while in motion and full enhancement were applied when motionstops and frame accumulation begins.

A small sample of the methods of an apparatus described herein are asfollows.

There is also described (A1) A method for operating an inspectionapparatus having an elongated inspection tube and an image sensor forgenerating image signals, said method comprising the steps of: (a)initiating a control signal to selectively output a frame of image data;(b) processing image data of one or more frames to determine a motionparameter; and (c) subsequent to initiation of said control signal toselectively output a frame, outputting a frame of image dataresponsively to said processing referred to in step (b). There is alsodescribed (A2) The method of claim A1, wherein said processing step (b)comprises processing less than a full frame of image data. There is alsodescribed (A3) The method of claim A1, wherein said control signal is afreeze frame control signal. There is also described (A4) The method ofclaim A1, wherein said control signal is a save frame control signal.There is also described (A5) The method of claim A1, wherein saidoutputting step includes the step of outputting a single frame. There isalso described (A6) The method of claim A1, wherein said processingincludes processing of frames of image data captured prior to a time ofinitiation of said control signal. There is also described (A7) Themethod of claim A1, wherein said processing includes processing offrames of image data captured subsequent to a time of initiation of saidcontrol signal. There is also described (A8) The method of claim A1,wherein said motion parameter is a parameter classifying a frame ofimage data as an “in motion” frame or a “motion free” frame. There isalso described (A9) The method of claim A1, wherein said motionparameter is a qualitative parameter indicating a degree of motion.There is also described (A10) The method of claim A1, wherein said frameof image data responsively output in step (c) is a noise reduced frameof image data provided by processing of several single frames. There isalso described (A11) The method of claim A1, wherein said frame of imagedata responsively output in step (c) is a noise reduced frame havingapplied thereto at least one of a set of non-uniform gain parameters anda set of non-uniform offset parameters, the at least one of a set ofnon-uniform gain parameters and a set of non-uniform offset parametersbeing provided for brightening darker areas of said noise reduced framewithout substantial or without any brightening of a bright spot of saidnoise reduced frame of image data. There is also described (A12) Themethod of claim A1, wherein said frame of image data responsively outputin step (c) is a noise reduced frame having applied thereto a set ofposition dependent non-uniform gain parameters determined responsivelyto a determination of regional brightness values in a frame of imagedata. There is also described (A13) The method of claim A1, wherein saidoutputting step includes the step of outputting a noise reduced filteredframe. There is also described (A14) The method of claim A1, whereinsaid outputting step includes the step of outputting a noise reducedfiltered frame, the noise reduced filtered frame being provided byprocessing of several frames. There is also described (A15) The methodof claim A1, wherein said processing includes processing of a pluralityof frames to determine a super frame. There is also described (A16) Themethod of claim A1, wherein said method includes the steps of detectingbrightness of incoming image data and responsively to a determinationthat brightness has fallen below a threshold brightness processing superframes in step (b) for determination of a motion parameter. There isalso described (A17) The method of claim A1, wherein said outputtingstep includes the step of repeatedly outputting a buffered framebuffered in a frame buffer to a display. There is also described (A18)The method of claim A1, wherein said outputting step includes the stepof outputting a buffered frame to a memory device.

There is also described (B1) An inspection apparatus comprising: anelongated inspection tube; (a) a two dimensional image sensor comprisinga plurality of pixels, the two dimensional image sensor generating imagesignals corresponding to light incident on said pixels; (b) a userinterface enabling a user to activate first or second configurations,the apparatus being adapted so that when said first configuration isactive, said apparatus selects a frame for outputting according to afirst algorithm, said apparatus further being adapted so that when saidsecond configuration is active said apparatus selects a frame foroutputting according to a second algorithm, wherein said secondalgorithm is different than said first algorithm; (c) wherein saidinspection apparatus is further configured to allow a user to initiate acontrol signal to selectively output a frame of image data; and (d)wherein said inspection apparatus subsequent to an initiation of acontrol signal to selectively output a frame selects a frame of imagedata for outputting in a manner that varies depending on whether saidfirst configuration or said second configuration has been selected.There is also described (B2) The inspection apparatus of claim B1,wherein said inspection apparatus, when said first configuration isactive, determines a binary motion parameter, and when said secondconfiguration is active, determines a qualitative motion parameter.There is also described (B3) The inspection apparatus of claim B1,wherein said user interface is a graphical user interface displaying adesignator for each of said first and second configurations.

There is also described (C2) An inspection apparatus comprising: (a) anelongated inspection tube; (b) a two dimensional image sensor comprisinga plurality of pixels, the two dimensional image sensor generating imagesignals corresponding to light incident on said pixels; (c) wherein saidinspection apparatus is configured to be capable of: (i) responding to auser-initiated control signal to selectively output a frame of imagedata; (ii) processing image data of one or more frames to determine amotion parameter; and (iii) selectively outputting a frame of image dataresponsively to said processing. There is also described (C2) Theapparatus of claim C1, wherein said control signal is a freeze framecontrol signal. There is also described (C3) The apparatus of claim C1,wherein said control signal is a save frame control signal. There isalso described (C4) The apparatus of claim C1, wherein said outputtingstep includes the step of outputting a single frame. There is alsodescribed (C5) The apparatus of claim C1, wherein said processingincludes processing of frame image data captured prior to a time ofinitiation of said control signal to selectively output a frame of imagedata. There is also described (C6) The apparatus of claim C1, whereinsaid processing includes processing of frame image data capturedsubsequent to a time of initiation of said control signal to selectivelyoutput a frame of image data. There is also described (C7) The apparatusof claim C1, wherein said motion parameter is parameter classifying aframe of image data as a motion frame or a motion free frame. There isalso described (C8) The apparatus of claim C1, wherein said motionparameter is a parameter indicating a qualitative level of motion. Thereis also described (C9) The apparatus of claim C1, wherein said frame isa noise reduced frame generated by processing multiple frames of imagedata. There is also described (C10) The apparatus of claim C1, whereinsaid processing comprises averaging multiple frames of image data. Thereis also described (C11) The apparatus of claim C1, wherein saidapparatus in outputting said frame outputs a noise reduced frameprovided by processing of several single frames.

There is also described (D1) A method for operating an inspectionapparatus of the type having an elongated inspection tube and an imagesensor generating image signals, said method comprising the steps of:(a) configuring said inspection apparatus to process a plurality offrames to provide a noise reduced frame of image data, (b) generating acontrol signal to selectively output a frame responsively to an actionby a user to initiate said control signal; (c) processing image data todetermine a motion parameter for a plurality of frames of image data;and (d) subsequent to generation of said control signal to selectivelyoutput a frame, outputting said noise reduced frame of image dataresponsively to said processing step (c). There is also described (D2)The method of claim D1, wherein said apparatus in executing saidprocessing step (c) and said outputting step (d) determines whetherseveral successive frames are motion free and outputs a said noisereduced frame if each of said several frames are determined to be motionfree. There is also described (D3) The method of claim D1, wherein saidconfiguring step comprises the step of configuring said inspectionapparatus to average a plurality of frames to provide a noise reducedframe of image data.

This written description uses examples to disclose the invention,including the best mode, and also to enable any person skilled in theart to make and use the invention. The patentable scope of the inventionis defined by the claims, and may include other examples that occur tothose skilled in the art. Such other examples are intended to be withinthe scope of the claims if they have structural elements that do notdiffer from the literal language of the claims, or if they includeequivalent structural elements with insubstantial differences from theliteral language of the claims.

1. An inspection apparatus, comprising: an image sensor comprising aplurality of pixels, wherein the image sensor is configured to generateimage signals based at least in part on light incident on each of theplurality of pixels; a processor communicatively coupled to the imagesensor, wherein the processor is programmed to: determine a first imageframe corresponding to first image signals generated by the imagesensor; determine a second image frame corresponding to second imagesignals generated by the image sensor; detect a first edge present inthe first image frame and the second image frame; align the first edgein the first image frame and the second image frame by offsetting thefirst image frame, the second image frame, or both; and determine afirst super frame after aligning the first edge based at least in parton a sum of the first image frame and the second image frame tofacilitate reducing noise in the first super frame compared to the firstimage frame and the second image frame.
 2. The inspection apparatus ofclaim 1, comprising: a base assembly, wherein the processor is disposedin the base assembly; and an inspection tube coupled to the baseassembly at a proximal end and the image sensor at a distal end.
 3. Theinspection apparatus of claim 1, wherein the processor is programmed to:determine a third image frame corresponding to third image signalsgenerated by the image sensor; determine a second edge present in thethird image frame and the first super frame; align the second edge inthe third image frame and the first super frame by offsetting the firstsuper frame; and determine a second super frame after aligning thesecond edge based at least in part on accumulation of the third imageframe with the first super frame to facilitate reducing noise in thesecond super frame compared to the third image frame.
 4. The inspectionapparatus of claim 3, wherein, to align the second edge, the processoris programmed to: determine a first pixel position of the second edge inthe first super frame; determine a second pixel position of the secondedge in the third image frame; and offset the first super frame based atleast in part on difference between the first pixel position and thesecond pixel position.
 5. The inspection apparatus of claim 1,comprising: a frame buffer communicatively coupled to the processor,wherein the frame buffer is configured to store image data correspondingwith the first super frame; and a memory device communicatively coupledto the processor; wherein the processor is programmed to store the imagedata from the frame buffer to a memory location in the memory devicewhen a save frame control signal is received.
 6. The inspectionapparatus of claim 1, comprising: a frame buffer communicatively coupledto the processor, wherein the frame buffer is configured to store firstimage data corresponding with an average image frame; and a displaycommunicatively coupled to the frame buffer; wherein the processor isprogrammed to: determine the first image data by averaging the firstimage frame and the second image frame; and instruct the frame buffer tocontinuously output the first image data to enable the display tocontinuously display the average image frame.
 7. The inspectionapparatus of claim 1, wherein, to align the first edge, the processor isprogrammed to: determine a first pixel position of the first edge in thefirst image frame; determine a second pixel position of the first edgein the second image frame; and offset only the second image frame basedat least in part on difference between the first pixel position and thesecond pixel position when the second image frame is captured after thefirst image frame.
 8. The inspection apparatus of claim 1, wherein theprocessor is programmed to: determine a third image frame correspondingto third image signals generated by the image sensor, wherein the firstedge is present in the third image frame; align the first edge in thefirst image frame, the second image frame, the third image frame byoffsetting the first image frame, the second image frame, the thirdimage frame, or any combination thereof; and determine the first superframe after aligning the first edge based at least in part on a sum ofthe first image frame, the second image frame, and the third image framethe second image frame, and the third image frame, or both to facilitatereducing noise in the first super frame compared to the third imageframe.
 9. The inspection apparatus of claim 1, wherein the processor isprogrammed to: determine a third image frame corresponding to thirdimage signals generated by the image sensor, wherein the third imagesignals are generated after the first image signals and the second imagesignals; determine a fourth image frame corresponding to fourth imagesignals generated by the image sensor, wherein the fourth image signalsare generated after the first image signals and the second imagesignals; determine a second edge present in the third image frame andthe fourth image frame; align the second edge in the third image frameand the fourth image frame by offsetting the third image frame, thefourth image frame, or both; and determine a second super frame afteraligning the second edge based at least in part on a sum of the thirdimage frame and the fourth image frame to facilitate reducing noise inthe second super frame compared to the third image frame and the fourthimage frame.
 10. The inspection apparatus of claim 9, wherein theprocessor is programmed to: determine a third edge present in the firstsuper frame and the second super frame, wherein the second super frameis determined after the first super frame; align the third edge in thefirst super frame and the second super frame by offsetting the firstsuper frame, the second super frame, or both; and determine a thirdsuper frame after aligning the third edge based at least in part onaccumulation of the second super frame with the first super frame tofacilitate reducing noise in the third super frame compared to the firstsuper frame and the second super frame.
 11. The inspection apparatus ofclaim 1, wherein the processor is programmed to: instruct the imagesensor to generate the first image signals and the second image signalsat a first brightness level; instruct the image sensor to generate thirdimage signals at a second brightness level different from the firstbrightness level; determine a third image frame corresponding to thethird image signals; and generate a high dynamic range image byutilizing at least a portion of the first super frame and at least aportion of the third image frame, wherein the high dynamic range imagecomprises more unsaturated pixels than the first super frame.
 12. Aremote visual inspection system, comprising: an imager comprising anarray of light-sensitive pixels, wherein the imager is configured togenerate image data based at least in part on light level sensed by eachpixel in the array of light-sensitive pixels; a processorcommunicatively coupled to the image sensor, wherein the processor isprogrammed to: determine a plurality of image frames each based oncorresponding image data generated by the image sensor; determine afirst baseline image by summing pixel values at a first one or morepixel locations in each of the plurality of image frames; determine afirst image frame based on first image data generated by the imagerafter the first baseline image is determined; determine a first pixellocation of a first sharp brightness transition in the first baselineimage; determine a second pixel location of the first sharp brightnesstransition in the first image frame; align the first baseline image withthe first image frame when the first pixel location and the second pixellocation differ by applying a first rotation of the first baseline imagea first translation of the first baseline image, or both on the firstimage frame; and determine a second baseline image by summing pixelvalues at a second one or more pixel locations in each of the pluralityof image frames and the first image frame after the first baseline imageis aligned to facilitate reducing noise in the second baseline imagecompared to the first image frame and the first baseline image.
 13. Theremote visual inspection system of claim 12, wherein the processor isprogrammed to: determine a second image frame based on second image datagenerated by the imager after the second baseline image is determined;determine a third pixel location of a second sharp brightness transitionin the second baseline image; determine a fourth pixel location of thesecond sharp brightness transition in the second image frame; align thesecond baseline image with the second image frame when the third pixellocation and the fourth pixel location differ by applying a secondrotation of the second baseline image, a second translation of thesecond baseline image, or both on the second image frame; and determinea third baseline image by summing pixel values at a third one or morepixel locations in each of the plurality of image frames, the firstimage frame, and the second image frame after the second baseline imageis aligned to facilitate reducing noise in the third baseline imagecompared to the second image frame and the second baseline image. 14.The remote visual inspection system of claim 12, wherein the processoris programmed to: instruct the imager to capture the plurality of imageframes and the first image frame at a first brightness level; instructthe imager to generate second image data at a second brightness leveldifferent from the first brightness level; determine a second imageframe based on the second image data; and generate a high dynamic rangeimage by utilizing at least a portion of the second baseline image andat least a portion of the second image frame, wherein the high dynamicrange image comprises more unsaturated pixels than the second baselineimage.
 15. The remote visual inspection system of claim 12, wherein theprocessor is programmed to: align the first baseline image with thefirst image frame only when the first pixel location and the secondpixel location differ by less than a threshold; and set the first imageframe as the second baseline image when the first pixel location and thesecond pixel location differ by more than the threshold.
 16. The remotevisual inspection system of claim 12, wherein the remote visualinspection system comprises an endoscope, a borescope, a pan-tilt zoomcamera, a push camera, or any combination thereof.
 17. A method foroperating a remote visual inspection system, comprising: instructing,using one or more processors, an image sensor of the remote visualinspection system to capture a plurality of image frames at a firstbrightness level; determining, using the one or more processors, a firstsuper frame by accumulating pixel values in each of the plurality ofimage frames; instructing, using the one or more processors, the imagesensor to capture a first image frame after the plurality of imageframes at the first brightness level; detecting, using the one or moreprocessors, whether first movement of the image sensor occurred betweencapture of the plurality of image frames and capture of the first imageframe; and when the first movement of the image sensor is detected:adjusting, using the one or more processors, the first super frame byrotating, translating, or both the first super frame to offset the firstmovement; and determining, using the one or more processors, a secondsuper frame by accumulating pixel values in each of the plurality ofimage frames with corresponding pixel values in the first image frame.18. The method of claim 17, comprising: when the remote visualinspection system receives a freeze frame control signal or a storeframe control signal before the first image frame is captured:determining, using the one or more processors, a first average imageframe by: summing a first pixel value from each of the plurality ofimage frames at a first pixel location to determine a first summed pixelvalue; and dividing the first summed pixel value by number of imageframes in the plurality of image frames; and instructing, using one ormore processors, a frame buffer of the remote visual inspection systemto output first image data corresponding with the first average imageframe; and when the remote visual inspection system receives the freezeframe control signal or the store frame control signal after the firstimage frame is captured: determining, using the one or more processors,a second average image frame by: summing a second pixel value from eachof the plurality of image frames and the first image frame at a secondpixel location to determine a second summed pixel value; and dividingthe second summed pixel value by the number of image frames in theplurality of image frames plus one.
 19. The method of claim 17, wherein:detecting whether the first movement of the image sensor occurredcomprises: determining a first pixel location in the first super frameat which a physical feature is present; determining a second pixellocation in the first image frame at which the physical feature ispresent; and determining that the first movement occurred when the firstpixel location and the second pixel location are different; andadjusting the first image frame comprises rotating, translating, or boththe first super frame based at least in part on difference between thefirst pixel location and the second pixel location.
 20. The method ofclaim 17, comprising: instructing, using one or more processors, theimage sensor of the remote visual inspection system to capture a secondimage frame; detecting, using the one or more processors, whether secondmovement of the image sensor occurred between capture of the pluralityof image frames and capture of the second image frame; and when thesecond movement of the image sensor is detected: adjusting, using theone or more processors, the second super frame by rotating, translating,or both the second super frame to offset the second movement;determining, using the one or more processors, a third super frame byaccumulating pixel values in each of the plurality of image frames withcorresponding pixel values in the first image frame and correspondingpixel values in the second image frame when the second image frame iscaptured at the first brightness level; and determining, using the oneor more processors, a high dynamic range image by utilizing at least aportion of the second super frame and at least a portion of the secondimage frame when the second image frame is captured at a secondbrightness level different from the first brightness level.